Compressing Relations and Indexes

We propose a new compression algorithm that is tailored to database applications It can be applied to a collection of records and is especially e ective for records with many low to medium cardinality elds and numeric elds In addition this new technique sup ports very fast decompression Promising application domains include decision sup port systems DSS since fact tables which are by far the largest tables in these applications contain many low and medium cardinality elds and typically no text elds Further our decompression rates are faster than typical disk throughputs for sequential scans in con trast gzip is slower This is important in DSS appli cations which often scan large ranges of records An important distinguishing characteristic of our algorithm in contrast to compression algorithms pro posed earlier is that we can decompress individual tu ples even individual elds rather than a full page or an entire relation at a time Also all the infor mation needed for tuple decompression resides on the same page with the tuple This means that a page can be stored in the bu er pool and used in compressed form simplifying the job of the bu er manager and improving memory utilization Our compression algorithm also improves index structures such as B trees and R trees signi cantly by reducing the number of leaf pages and compressing in dex entries which greatly increases the fan out We can also use lossy compression on the internal nodes of an index